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(54) Abnormality supervising system 



(57) The system compares an input picture with a previously stored reference picture having no 
abnormality, processes the difference signal and sends the processed picture signal to an abnormality 
discrimination means 12 having preliminarily stored information for abnormality discrimination. Upon 
presence of an abnormality, the means 12 operates an output means 15 for informing operation, whereby 
the abnormality discrimination means can effectively attain the abnormality discrimination on the basis of 
the preliminarily stored information, in a highly reliable manner. 

The system can be used to detect the presence and movement of an intruder, or the presence of a fire 
in a monitored zone. The system is designed so that it only responds to certain abnormal events 
represented by the data stored in a knowledge base 1 3 of the abnormality discrimination means. 
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Abnormality supervising system 

This invention relats to abnormality supervising systems employing picture input means including 5 
such picture pickup means as TV cameras and the like and, more specifically, to an abnormality 
supervising system of a picture recognition type in which an input picture obtained by the 
picture pickup means with respect to a predetermined monitoring zone is processed to detect 
absence or presence of abnormality occurring in the zone. 

The abnormality supervising system of the type referred to contributes effectively to ' crime iu 
prevention of intrusion into private houses and grounds, burglary to art galleries or exhibition 
halls and so on, and is also effectively employed as a fire protecting system for detecting fire 
occurrence in residential houses, office buildings, factories and the like, or as a safety system 
for preventing any accident in such specific areas as factories due to any abnormality occur- 

15 ""H^ been suggested for example, an abnormality supervising system in which a lumi- 
nance difference between corresponding picture elements of an input picture obtained through, 
for example, a picture pickup means and of a previously prepared reference picture indicative of 
a normal state of the monitoring zone is obtained and converted to a binary signal and then the 
20 number of the picture elements of the luminance difference exceeding a set level is counted. 

According to this system, a certain large number having reached by such picture elements in the | 
luminance difference more than the set level is discriminated as an occurrence of a remarkable 
change in the monitoring zone of the picture pickup means, and thus as an abnormality taken 
place in the zone, and, when such large number exceeds a predetermined value, the abnormality 
25 occurrence is informed by an alarm sound or the like. This system, however has had a problem 
that since the discrimination of abnormality occurrence is made only on the basis of the 
luminance difference between the input and reference pictures, even a variation in the luminance 
caused due to a shaking of tree located in the monitoring zone, falling rain or snow, lightning or 
the like is informed as the abnormality occurrence. 
30 Also disclosed in U.S. Patent No. 4,249,207 to R. K. Harman et al is a superv.sory system in 30 
which a detection zone is set to lie between two parallel fences, the detection zone is divided 
into an array of cells each enclosing the monitored image of a man in response to his varying 
distance, and input video image for each cell is digitalized to discriminate variations in the light 
level of the respective images. With this system, an object moving at a certain speed can be 
35 discriminated by means of a filtering in time, while an object considerably larger or smaller than 35 
each cell can be discriminated by means of spatial filtering. Therefore, this superv.sory system 
can be arranged to discriminate an abnormally moving object from a normally moving object. 
However, this arrangement still involves a problem that even a man abnormally moving without 
any criminal intent of theft or like is grasped. In other words, the system has been defective in 
40 that its discriminating measure for any abnormally moving object is insufficient and is thus unaDle 
to precisely discriminate abnormality from normality sufficiently satisfactorily. 

A primary aim of the present invehtion is, therefore, to provide an abnormality supervising 
system which can discriminate moving attitude of an object within a zone monitored by a picture 
input means with a high precision to realize discrimination between normal and abnormal states 
45 sufficiently satisfactorily, for remarkably improving the reliability. 

According to the present invention, this aim is attained by providing an abnormality supervis- 
ing system wherein input pictures of a monitoring zone obtained by a picture input means are 
compared with a reference picture, the pictures are processed by a picture processing means to 
obtain information necessary for discriminating the abnormality, and the abnormality is discrimi- 
50 nated on the basis of the thus obtained information, the system being characterized in compris- oo 
ing means for previously storing therein the information necessary for the abnormality discrimina- 
tion to be compared with the information obtained from the picture processing means, and 
means for discriminating the abnormality on the basis of the- information obtained fropm the 
picture processing means and the information in the storing means. 
55 Other aims and advantages of the present invention shall be made clear in the following oo 
description of the invention detailed with reference to preferred embodiments shown in accom- 
panying drawings, in which:- 

Figure 1 is a block diagram of a basic embodiment of the abnormality supervising system 

according to the present invention; . ft _ 

60 Figure 2 is a flowchart showing the processing algorithm of a picture processing means in the e>u 

system of Fig. 1; 

Figure 3 is a diagram for explanation of the usage of the system of ng. i; 
Figure 4 is a block diagram of a practical embodiment of the abnormality supervising system 
according to the present invention; t c- a fi^ 

65 Figure 5 is a diagram for explanation of the usage of the system of Fig. 4; 
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Figure 6 is a block diagram at a major part of another embodiment of the system according to 
the present invention; . 
Figures 7 and 8 are diagrams for explaining the usage of the system of Fig. 6, respectively; 
Figure 9 is a block diagram at a major part of another embodiment of the system according to 
5 the present invention; 

F/gure 70 is a diagram for explaining the usage of the system of Pig. 9; 

Figure 1 1 is a block diagram of another embodiment of the system according to the present 
invention; 

F/gure 72 is a diagram for explaining the usage of the system of Fig. n, 
10 F/gure 13 is a block diagram at a major part of another embodiment of the system according iu 
to the present invention; 

Figure 14 is a diagram for explaining the usage of the system of Fig. 13; 
Figure 15 is a block diagram at a major part of another embodiment of the system according 
to the present invention; * c iC ir 

15 Figure 16 is a diagram for explaining the usage of the system of Fig. lb; 

Figure 7 7 is a block diagram of another embodiment of the system according to the present 
invention; 

Figure 18 is a diagram for explaining the usage of the system of Hg. i /; 
Figure 19 is a block diagram of another embodiment of the system according to the present 

20 inv *™™' 2() js g b|Qck diagram at a ma jor part of another embodiment of the system according 



to the prsent invention; 

F/gure 2 7 is a timing chart for explaining of the operation of the system of Fig. 2U, 
F/gure 22 is a block diagram at a major part of another embodiment of the system according 
25 to the present invention; 

F/gures 23 and 24 are flowcharts of further different embodiments of the system according to 
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the present invention; 

F/gure 25 is an installation diagram of a TV camera used in system of Fig. 24; 

Figure 26 is a diagram for explaining relationship between the coordinate on monitoring video 
30 screen and actual distance of monitoring object in the system of Fig. 24; 

Figure 27 is a block diagram of another embodiment of the system according to the present 
invention; 

Figure 28 is a flowchart of threshold calculation in the system of Hg. £/, 

Figure 29 is a block diagram of another embodiment of the system according to the present ^ 
35 mutton. ^ ^ ^ ^ diagrams of other different embodiments of the system according 

to the present invention; 

F/gures 32 and 33 are block diagrams of furter different embodiments of the system according 

• to the present invention; . AO 

40 Figure 34 is a block diagram at a major part of still another embodiment of the system 4U 
according to the present invention; 

Figure 35 shows an example of an input picture in the system of Fig. 34; 
Figure 36 shows an example of memory contents in the system of Fig. 24; 

Figures 37 and 38 are block diagrams of other different embodiments of the system according ^ 
45 to the present invention; 

F/gure 39 is a block diagram of another embodiment of the system according to the present 

invention; OQ 
Figure 40 is a diagram showing a picture pickup state of the system of Hg jy, 
Figure 41 is a diagram for explaining the operation of the system of Fig. 39; 
50 Figure 42 is a diagram for explaining the operation of the foregoing embodiments as shown in 50 

the same manner as in Fig. 41; 

Figure 43 is a schematic block diagram of another embodiment of the system according to the 

present invention; 

Figure 44 is a diagram for explaining the operation of the system ot Ng. 
55 Figure 45 is a diagram for explaining the operation of the foregoing embodiments as shown in bb 
the same manner as in Fig. 44; ,. . 

Figure 46 is a block diagram of another embodiment of the system according to the present 

invention; . . . A . 

Figure 47 shows an example of a reference picture for comparison with an input picture in the ^ 

60 system of Fig. 46; . . 

F/gures 48 faj fo ffl are diagrams for explaining the operation of a texture operating means in 

the system of Fig. 46; 

Figure 49 is a block diagram of another embodiment of the system according to the present 
65 ^Figure 50 is a diagram showing an installation of monitoring TV cameras in the system of Fig. 65 
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Figures 51 and 52 show different examples of monitoring picture in the case of Fig. 49; 
Figure 53 is a block diagram of another embodiment of the system according to the present 

5 ^Figure^SA is a diagram for explanation of renewing operation of a reference picture in the 

*^wesSl?toS8 are schematic block diagrams of other different embodiments of the system 

according to the present invention; 
Fiqure 59 is a diagram for explanation of the operation of the system of Fig. 68; 
10 Figures 60 to 63 are block diagrams of further different embodiments of the system according 

to the present invention; 

Figures 64 to 66 are diagrams for explanation of the operation of the system of Fig. 63, 
Figure 67 is a block diagram of a coordinate conversion section in the system of Fig. 63; 
Figure 68 is a diagram for explaining anoter form of the operation of the system of Fig. 63, 
15 Figures 69 and 70 are block diagrams of other different embodiments of the system according 

to the present invention; _ 
Figures 71 to 75 are diagrams for explanation of the operation of the system of Fig. 70, 
Figure 76 is a block diagram of another embodiment of the system according to the present 

invention; , , , -, e . o<-» 

20 Figure 77 is a diagram for explanation of the operation of the system of Fig. 7b, ^ 
Figures 78 and 79 are block diagrams at their major part of other different embodiments ot \ 

the system according to the present invention; .. 

Figure 80 is a block diagram of a major part of another embodiment of the system according 
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to the present invention; ... on 9 c 

25 Fiaure 81 is a diagram for explaining the operation of the system of Fig. BU; 

Figure 82 is a block diagram at a major part of another embodiment of the system according 

to the present invention; .. t . „___«. 

Figure 83 is a block diagram of another embodiment of the system according to the present 

30 ^Figures' 84 (a) to (e) are diagrams for explanation of the operation of the system of Fig. 83; 30 
Figure 85 is a diagram for explaining the operation of a picture processing section in the 

SV Sg™e°S6 ^ a block diagram of another embodiment of the system according to the present 

35 ' nV F/gure n 's7 shows a practical explanatory diagram for showing a case where the present 35 
invention is applied to a intruder supervising system; 

Figure 88 schematically show an interior arrangement of the supervisory system of Fig. 8/; 

^Figures 89 to 96 show explanatory views of examples of practical application of the abnormal- 
40 itv supervising system according to the present invention. 

While the present invention shall now be described with reference to the preferred embodi- g 
ments shown in the drawings, it should be understood that the intention is not to limit the 1 
invention only to the particular embodiments shown but rather to cover all alterations, modifica- 
tions and equivalent arrangements possible within the scope of appended claims. 

45 Referring to Fig. 1 , there is shown an abnormality supervising system according to the present «d 
invention, which comprises a picture input means 10 which may be one of such picture pickup 
means as visual and infrared range TV cameras including vidicon, CCD and the like, or preferably 
an infrared TV camera of a pyroelectric vidicon type specifically useful in detecting intrudcers, 
fire and the like. As the picture pickup means, further, a color TV camera, wireless TV camera 

50 of wireless picture signal transmission type or the like may be used. A mon.tor.ng-area picture 50 
signal picked up by the picture input means 10 is converted to a digital signal in the input 
means 10 and then sent to a picture processing means 11. 

In the picture processing means 1 1 , as will be clear from Fig. 2 showing a picture processing 
algorithm, an inter-picture-element subtraction is first performed between momentarily varying 

55 input picture of monitoring zone received from the picture input means 10 as A/D converted sa 
therein and such a reference picture containing no abnormality signal in respect of the same 
monitoring zone that has been obtained by storing previous picture in normal state, whereby a 
converted picture in which only picture elements showing any change in their luminance are 
provided with a certain value will be obtained. The picture thus subjected to the subtraction is 

60 then subjected to a filtering processing with use of, for example, a 3x3 mask to reduce or 60 
eliminate noise. Next, the respective picture elements are sliced with predetermined upper and 
lower limits the picture elements in a predetermined range are converted into binary signals, 
which signals are again filtered for noise elimination, and then the picture of the binary signals 
are labeled. From respective objects in the labeled picture, any object having a predetermined 

65 area i e a picture element number less than predetermined is removed while other objects of 
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the picture elements more than the predetermined area are calculated with respect to such 
feature values as centroid position, two-dimensional moment and the like. This picture process- 
ing procedure is executed for each input picture frame, and thus processed object is subjected 
to a frame tracking and is provided, together with a warning level value later described in the 
5 monitoring zone, to an abnormality discrimination means 12 of a unique arrangement in the 5 
present invention. 

The abnormality discrimination means 12 forms a so-called expert system, that is, the means 
is so provided that a deduction means 14 discriminates the presence or absence of an abnor- 
mality on the basis of information from a knowledge base 13, the information having been 

10 obtained from a viewpoint of crime preventive supervision and preliminarily provided in the base 10 
13. Referring more in detail with reference to Fig. 3, it is here assumed that a house window is 
being monitored by an externally installed TV camera in a framing as shown, with differently 
ranked warning levels according to the information of the knowledge base 13 in such that an 
outer peripheral area shown as hatched of the window is set to have a warning level of 1 and 

15 an inner area of the window itself is to have a warning level of 2, while other area than these 15 
two is given a warning level of 0, the larger number of the warning level demanding a higher 
warning degree. In this case, the knowledge base 13 stores as its knowledge many rules 
according to which the discrimination is so made that, with movement with time of any 
monitored moving object employed as a parameter, the monitored object is determined to be a 

20 dweller or a passerby who shows a normal moving pattern or to be an intruder of an abnormal 20 
moving pattern, and an information on the window obtained by monitoring it as seen in Fig. 3 
and through the picture processing means 1 1 is subjected to the discrimination of the absence 
or presence of any abnormality. It should be appreciated that the discrimination rules can be of 
various sorts, one of which, for example, will be that, when the monitored object is present only 

25 within the area having the warning level 2, the object is judged to be a dweller, while an object 25 
moving sequentially from the area of warning level 0 through the level 1 area to the level 2 area 
and staying in the level 2 area is judged to be an intruder. — 

Further, in an event where the window of Fig. 3 to be monitored involves a tree or shrub 
which provides a dead angle to the TV camera, the picture pickup means of the picture input 

30 means 10 may include two or more TV cameras. When it is desired to keep the supervising 30 
system operativeonly in the nighttime, the system can be operatively associaed with a light 
sensor, timer or the like to realize the supervision only for a desired time zone. In addition, the 
abnormality supervising system according to the present invention may be operatively associated 
with a human body sensor to employ a discriminating rule for their associated operation. 

35 When the abnormality discrimination means 12 discriminates that an abnormality is present, no 35 
means 12 provides an abnormality discrimination output to an output means 15 which causes, 
for example, a portion in which the abnormality is present flickered on a monitoring video screen 
or a voice signal generated, for an alarm operation. The output means 15 also may cause the 
abnormality portion color-displayed, the object's abnormal movement displayed in locus, or 

40 location and time of the abnormality recorded. Further, the output means 15 may even be 40 
designed to execute a wireless transmission of an abnormality informing signal or of an image 
showing the abnormality. 

Such setting of the areas of different warning levels 1 and 2 in the monitoring zone as shown 
in Fig. 3 can be performed before placement of the system into its monitoring state by using a 

45 light pen, cursor or the like with respect to the monitoring video screen, and this warning area 45 
setting can be achieved by using such a means as a graphic tablet based on a video image, 
photograph or the like of the monitoring zone. 

There is shown in Fig. 4 a practical abnormality supervising system embodying the basic 
system of Fig. 1, wherein the same constituent parts as those in Fig. 1 are denoted bg the 

50 same reference numerals but added by 10. More particularly, as will be clear from Fig. 4, an 50 
abnormality discrimination means 22 realizes the algorithm explained with reference to Fig. 2, 
that is, receives an output of a picture processing section 21 including a reference picture 
memory 21a, an input picture memory 21b and a picture processing means 21c as well as an 
output of a detection area memory 27 receiving an output of a detection area setting means 26 

55 which is provided to divide the monitoring zone into several areas of different warning levels 55 
according to the demanded warning degree as shown in Fig. 3. A monitoring zone 26a of Fig. 5 
may be divided into three areas having sequentially increasing warning levels 1 to 3 by, for 
example, drawing the aras with a light pen based on a reference picture, or into four or more 
areas. The warning area information set by the detection area setting means 26 is stored in a 

60 detection area memory 27 so that the abnormality discrimination means 22 provides to an 60 
output means 25 an output corresponding to the warning degree or level on the basis of a 
luminance change component of the input picture with respect to the reference picture from the 
picture processing means 21, that is, the abnormal information and stored contents about the 
warning levels in the detection area memory 27. 

65 The output means 25 also receives an output of a warning level setting memory 28 which 65 
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stores information necessary to provide different warnings according to the warning levels, 
whereby the output means 25 can allow such informing operation as different alarm sound 
generation responsive to the warning levels, and the like operation. 
The abnormality supervising system according to the present invention can be also used in 
5 detecting occurrence of any abnormality in factory production line by means of monitoring lamps 
indicative of operating states of machines or the like installed in factories. Referring to Fig. 6. in 
which the same major constituent parts of this embodiment as those in the embodiment of Hg. 
4 are denoted by the same reference numerals but added by 10 and other constituent parts not 
illustrated in Fig. 6 may be the same as the corresponding parts in the embodiment of Fig. 4, an 
10 output of a variation pattern memory means 39 is provided to an abnormality discrimination 
means 32, in addition to outputs of a picture processing section 31 including a picture process- 
inq means 31c and of a detection area memory 37. The present embodiment is arranged so 
that when a variation with respect to the reference picture takes place in a predetermined ara ot 
the input picture, the pattern memory means 39 changes its memory content to conform it to 
15 the variation, and an output means 35 is thereby actuated to perform an informing operation. 
Assuming, for example, that such detection areas as shown by dotted lines in Fig. .re set in 
correspondence to an array 40 of lamps indicative of operating states of various ma < anes in a 
factory, and that simultaneous flickering of first and third lamps in the array 40 indices an 
abnormal state, this should be so stored in the variation pattern means 39 that an aonormality 
20 occurrence in the production line can be accordingly informed. 

The variation pattern memory means 39 can be effectively used also for supervising an 
intruder. To this end. such two-level warning supervision set for the house window as in Fig. J 
is assumed to be performed with respect to a residential ground, and the warning jevel is so set 
that as in Fig. 8. substantially the whole ground area including a house is set to have a warning 
25 level of 1 while an inner restricted area covering the house and immediately house-surroundings 
is to have a warning level of 2, and the variation pattern memory means 39 is so provided as 
to store a pattern which determines that objects moving sequentially from the warning level 1 
area to thelevel 2 area only through a predetermined gate part of the level 1 area are normal 
whereas objects moving from the level 1 area to the level 2 area not through the gate part are 
30 abnormal, for an effective informing operation. 

Referring to Fig. 9, there is shown a system in another embodiment, in which a plurality of 
detectioin area memories 47a to 47n are connected in parallel between a detection area setting 
means 46 and abnormality discriminating means which are similar to those in the embodiment of 
Fig 4, and these memories 47a to 47n are connected at their output ends to a switching 
35 means 50. In the present embodiment, different warning demand levels are allocated to divided 
monitoring time zones, and different warning levels ranked in correspondence to the warning 
demand levels of the time zones are stored respectively in each of these detection area 
memories 47a to 47n. The switching means 50 is arranged to use or select one of contents 
stored in the plurality of detection area memories 47a to 47n in response to such external signal 
40 as a clock signal generated by a digital clock or timer at every set time, or i "umination intensity 
signal sent from an illuminometer for measuring how the monitoring zone is light When the 
system is used to supervise an art gallery, museum, exhibition hall or the like where the same 
monitoring zone is to be supervised with different warning levels of several ranks depending on 
the respective time zones in which the gallery is open and is closed in such that, as shown in 
45 Fig. 10. only a limited area 46a covering articles being exhibited will be watched with several 
warning levels during the gallery opening time, but the entire gallery interior will be watched also 
with several warning levels during the gallery closed time with, for example, a gallery passage 
way 46b made to have a warning level of 0 during the gallery opening time but to have a 
warning level of 1 during the gallery closed time, the system allows a satisfactory result to be 

50 obtained. 

Shown in Fig. 1 1 is another embodiment in which an abnormality discrimination means 6Z 
receives an output of a picture input means 60 through a picture processing section 61 which 
includes a reference picture memory 61a, an input picture memory 61b and a picture processing 
means 61c, and also receives an output of a variation pattern memory means 71. In the present 

55 instance, the variation pattern memory means 71 holds therein luminance variations of abnormal 
time as storage contents so that, when the output pattern of the picture processing section 61 
corresponding to a luminance variation between the reference and input pictures coincides with a 
variation pattern of the memory means 71, an output means 65 issues an abnormality informing 
output. According to the present embodiment, therefore, in addition to such d 15 " 1 ^"^"^^ fin 

60 means of the output of the picture processing means 61 provided to the abnormality d.scr.mina- 60 
tion means 62 upon the luminance variation exceeding a certain threshold level as in the 
foregoing embodiments, a discrimination of abnormality or normality is executed according to the 
pattern of the output upon excess of the luminance variation over the threshold level, for a 
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further improved supervisory accuracy. fic . 
65 When the monitcmg zone is set. for example, with respect to an entrance door 72 ot a 
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house or building as shown in Fig. 12 and a normally flickering lamp 73 is provided immediately 
above the door 72, the picture processing section 61 issues a variation output to the abnormal- 
ity discrimination means 62 but, so long as this output is of a pattern not stored in the variation 
pattern memory means 71, then the output means 65 performs no informing operation. In other 
5 words the variation pattern memory means 71 may be so arranged as to preliminarily store the 5 
luminance variation upon opening of the door 72 and to issue an information output from the 
output means 65. Other arrangements and operation of the embodiment of Fig. 11 are substan- 
tially the same as those of the foregoing embodiments. 

Referring to Fig. 13, there is shown another embodiment in which outputs of a picture 
10 processing means 81 as well as a plurality ("n") of variation pattern memory means 91a to 91n 10 
are provided to a similarity operating means 82a which forms part of an abnormality discrimina- 
tion means. This operating means 82a compares the luminance variation output of the picture 
processing means 81 with respective pattern outputs of the "n" variation pattern memory 
means 91a to 91n and sends to a comparison means 82b the similarity value of the variation 
15 with respect to the most similar one of the "n" variation patterns. A predetermined threshold lb 
level is being provided to the comparison means 82b so that, when the luminance variation 
output is similar to one of the "n" variation patterns, the output is determined as not exceeding 
the threshold level and thus being normal, whereas any variation output not similar to any one of 
the "n" variation patterns is determined as exceeding the threshold level and thus being abnor- 
20 mal so that the comparison means provides an output to a subsequent-stage output means. 20 
When the system of Fig. 13 is used with, for example, a machine 92 a constituent member 
93 of which reciprocates along a rail 94 as shown in Fig. 14, the system can operate in such 
that so long as forward, backward (including every moment of displacement), halt and like 
operations of the member 93 are normal, any slight displacement of the member 93 due to a 
25 load imposed to the machine allows the luminance variation output to be similar to one of the <£5 
variation patterns of the variation pattern means 91a to 91 n so as not to have any abnormality 
informing output provided by the output means, whereas an irregular movement of the member 
93 on the rail 94 due to any trouble in the machine 92 is determined to be abnormal. Other 
arrangements and operation of the embodiment of Fig. 13 are substantially the same as those of 
30 the foregoing embodiments. 

In a further embodiment shown in Fig. 15, picture memories 112a to 112n are inserted in 
parallel with each other between a picture processing means 101 and an abnormality discrimina- 
tion means 102, and an output of a variation pattern memory means 1 1 1 is provided to the 
discrimination means 102. In this case, the picture memories 112a to 112n respectively store 
35 each of patterns of the luminance variation on the monitoring picture with time elapsing so that, 3b 
when a variation output of the picture processing means 102 coincides with one of these 
patterns of the picture memories 1 12a to 1 12n, an output is provided to the abnormality 
discrimination means 102, coincidence of which output with contents stored in the variation 
pattern memory means 1 1 1 causes an abnormality discrimination output provided to the subse- 

40 quent-stage output means. u - , 

In employing the system of Fig. 15 for monitoring, for example, running vehicles on a street 
crossing 1 13 as shown in Fig. 16 in which it has been so far impossible to judge, on the basis 
of only a momentary input picture from a picture input means, whether a car located at a 
position 1 14 has straightly crossed the crossing 113 from a position 1 15 or has turned right as 

45 shown by an arrow, the system according to the present invention makes this judgement 45 
possible in such that a picture of the car varying with time is sent to the picture memories 112a 
to 112n so that, when the varying picture does not coincide with any one of the picture 
memories 112a to 11 2n, a variation output is provided to the abnormality discrimination means 
102 Since the discrimination means 102 is provided with the output of the variation pattern 

50 memory means 1 1 1 having the same sorts of storage contents as that in the embodiment of 50 
Fig 6 coincidence of the varying picture sent to the discrimination means 102 with the variation 
pattern an abnormality discrimination output is sent to the output means. When the turning right 
of the car at the crossing 1 13 shown in Fig. 16 is not allowed legally, the system of Fig. 15 
can supervise the illegal right turning and inform it to a supervising officer. Other arrangements 

55 and operation of the embodiment of Fig. 15 are substantially the same as those of the foregoing 55 
embodiments. 

Referring to Fig. 17, there is shown an embodiment in which, as will be seen in its compari- 
son with, for example, Fg. 4, an attribute area memory 128 and a data base 129 are inserted 
between an area setting means 126 and an abnormality discrimination means 122. In this case, 

60 the attribute area memory 128 stores such attribute areas as shown by dotted lines in Fig. 18 SO 
with respect to a tree 126b and house 126c in a monitoring zone 126a set for a residential 
ground apart from the divided detection areas for the different warning levels, noticing in 
particular that a shake in the tree 126b or a flickering in illumination of the house 126c caus es 
the luminance variation in the input picture but a monitoring object coming behind the tree 126b 

65 causes no luminance variation. The data base 129 stores characters corresponding to the 65 
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respective attribute areas with respect to each of them stored in the attribute area memory 128. 
Since the knowledge stored in the data base 129 is provided to the abnormality discrimination 
means 122 together with outputs of a detection area memory 127 and of a picture processing 
means 121, any error resulting from the luminance variation caused by the shake of the tree or 
5 by the flickering of the house illumination or the absence of such variation caused by the object 5 
coming behind the tree within the monitoring zone can be compensated for. Other arrangements 
and operation of the embodiment of Fig. 17 are substantially the same as those of the foregoing 
embodiments. . . . , 

In an embodiment shown in Fig. 19, an output of an auxiliary sensor 136 is provided to an 
10 abnormality discrimination means 132, as will be clear when compared with the arrangement of 10 
Fig. 1. As the auxiliary sensor 136, such a human body sensor as an infrared ray sensor, an 
ultrasonic sensor or the like may be employed to detect the object coming behind the tree 
within the monitoring zone of Fig. 18, thus improvind the monitoring accuracy. Other arrange- 
ments and operation of the embodiment of Fig. 19 are substantially the same as those of the 
15 foregoing embodiments. 

Shown in Fig. 20 is another embodiment, in a picture processing section 141 of which a 
memory transfer circuit 141 d is inserted between an input picture memory 141b and a reference 
picture memory 141a, an output of the latter of which is provided to a picture processin means 
141c. The memory transfer circuit 141d receives an output of an AND circuit 141f which in turn 
20 receives an output of a timer 141 e. Also provided to the AND circuit 141f is an output of a 20 ^ 
NOT circuit 141g which receives an output of an abnormality discrimination means 142. Refer- 
ring also to Fig. 21, an input picture is applied to the input picture memory 141b at a cycle 
shown in Fig. 21a. When the picturse processing means 141 generates no variation output of a 
predetermined level, the output of NOT circuit 141g is applied to the AND circuit 141f which in 
25 turn sends a transfer command signal to the memory transfer circuit 141d in response to each 25 
of outputs from the timer 141e to thereby transfer every picture from the input picture memory 
141b to the reference picture memory 141a, whereby the reference picture in the reference 
picture memory 141a is renewed at a cycle shown in Fig. 21b and applied to the picture 
processing part 141c until the reference picture is renewed next, so that the latest normal 
30 reference picture will be obtained. When the abnormality discrimination means 142 discriminates 30 
the output of the picture processing means 141 to be abnormal, the means 142 generates an 
output. Accordingly, the output of the NOT circuit 141g is not provided to the AND circuit 141f 
and no transfer command signal is supplied from the AND circuit 141f to the memory transfer 
circuit 141d. As seen in Figs. 21c and 21d, therefore, the reference picture in the reference 
35 picture memory 141a is not renewed upon receipt of the abnormality discrimination output from 35 
the discrimination means 42. 

In this abnormality supervising system of Fig. 20, in contrast to the case of the foregoing 
embodiments using as the reference picture the input picture entered at every interval of a 
relatively long time, the abnormality discrimination can be realized without failing to notice the 
40 luminance variation of a gradually moving object. Other arrangements and operatic of the em- 40 
bodiment of Fig. 20 are substantially the same as those of the foregoing embodiments. 

In another embodiment of Fig. 22, the renewal of reference picture is carried out with a higher W 
reliability. That is, as will be clear when compared with Fig. 21, a picture-element average 
operating circuit 15 1h is inserted between an input picture memory 151b and a memory transfer 
45 circuit 151d, while an outpt of a timer 151e is independently provided to the circuit 151d, in a 45 
picture processing means 151. Also provided to the operating circuit 151h is an output of an 
AND circuit 151f which receives an output of an abnormality discrimination means 152 through 
a NOT circuit 151g, while the output of the timer 151e is provided to the AND circuit 151f 
through a monomultivibrator 151i. In the present embodiment, the output of the timer 151e is 
50 provided to the AND circuit 151f through the monomultivibrator 151i and the output pulse width 50 
of the timer 151e is expanded by the monomultivibrator 151i, so that a constant cycle output 
having a certain time width will be provided by the monomultivibrator 1 51 i to the AND circuit 
151f. In the illustrated embodiment, the timer pulse width is set to be, for example, an integer 
multiple of the input picture grasping cycle, and the average opeating circuit 151h is operated in 
55 response to the output of the AND circuit 151f. When the pulse width is set to be 5 times as 55 
large as the input picture grasping cycle, therefore, the average operating circuit 151h averages 
five input pictures, so that an average picture of five input pictures is provided as a new 
reference picture to the reference picture memory 151a at intervals of every five input picture 
grasping cycles, and this renewing is carried out normally at intervals of several minutes so that 
60 the gradually moving object within the monitoring zone can be reliably grasped. Other arrange- 60 
ments and operation of the embodiment of Fig. 22 are substantially the same as those of the 
foregoing embodiments. 

Referring to Fig. 23, there is shown an embodiment in which an area discriminating function 
for an object varying in its luminance is additionally provided to, for example, the picture 
65 processing means of Fig. 1. That is, in a picture processing means 161, an input picture is 65 
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provided to a reference picture memory 161a at intervals of a renewing period t = nT (T being 
the reference picture grasping cycle) to be subjected to an inter-picture-element subtraction with 
respect to an input picture from an input picture memory 161b. When a luminance difference 
obtained through the subtraction exceeds a predetermined value, a corresponding luminance 
5 variation is converted to a binary picture and then labeled. Then, the number of elements of 5 
such labeled pictures in each cluster is counted, that is, the area of each cluster is calculaed and 
then compared with a present threshold area. When the is such a cluster that has an area 
satisfying an expression S K l<Sj<S H , wherein S L is the lower threshold value of the set area, S H 
is the upper threshold value of the set area and Sj is the area of an i-th cluster, an abnormality 

10 output signal is issued. With the above area discriminating arrangement of the present embodi- 10 
ment, any luminance variation only due to the shake of a tree located within the monitoring 
zone,' falling rain or snow, flickering of illumination or the like will not be discriminated as being 
an abnormality, so as to effectively prevent any erroneous operation. Other arrangements and 
operation of the embodiment of Fig. 23 are substantially the same as those of the foregoing 

15 embodiments. 15 
An embodiment shown in Fig. 24 has an arrangement for discriminating the area of an object 
with a higher reliability. As will be clear when compared with Fig. 23, the number of elements in 
each cluster of the labeled picture is counted to calculate the cluster area as well as its centroid, 
the threshold value is operated according to the centroid coordinates for each. cluster, and it is 

20 judged whether or not the area satisfies the expression S L <Si<S H in the same manner as in the 20 
embodiment of Fig. 23. With a picture pickup TV camera TVC of an input picture means 
installed, for example, at a high position as directed obliquely downwardly to secure a broad 
monitoring zone as shown in Fig. 25, it normally happens that an object closer to the camera 
TVC is monitored to be larger on the video screen but is monitored to be smaller when remote 

25 from the camera through the object per se does not change its size, but the present embodi- 25 
ment used in such situation can effectively correct such magnitude difference on the video 
screen between the pictures of the identical object located close to and remote from the 
camera. 

The correcting operation of the present embodiment will be detailed with reference to Figs. 2b 

30 and 26. A distance R 0 between the vertical position of the picture pickup camera TVC on the 30 
ground surface and an intersecting point of the optical axis of the camera with the ground 
surface is found in accordance with an equation R o = H.cosec0, where H is the installation height 
of the camera TVC and 6 is an angle defined by the optical axis and the ground surface. Where 
the visual field angle of the camera TVC is a, an actual upper limit distance R H as well as an 

35 actual lower limit distance R L of the monitored picture can be obtained by mans of equations 35 
R H =H.cosec«9-a/2) and R L ==H.cosec(0+a/2), respectively. When it is assumed as shown in Fig. 
26a, on the video screen that an X axis is taken to intersect the optical axis of the camera and 
the X coordinate values of the lower and upper monitoring screen limits are 0 and A, respec- 
tively/, an actual distance R corresponding to a point on the screen is found in accordance with 

40 an equation R = H.cosec[0-a|X/{A- 1/2)}]. Since the size of an object on the screen of the 40 
monitoring zone is reverse proportional to the square of the actual distance, the magnitude 
difference on the screen between the pictures of the identical object located close to and remote 
from the camera is corrected, in the area comparison, by multiplying by 1/R 2 the lower and 
upper limit threshold values S L and S H of the set area on the basis of the calculated centroid 

45 position for each cluster and then operating the equation S L <S,<S H . In practice, the picture 45 
processing means may preliminarily be provided with a memory which stores such a conversion 
table of coordinate/distance correction coefficients as presented in Fig. 26b. Other arrangements 
and operation of the embodiment of Fig. 24 are substantially the same as those of the foregoing 
embodiments. 

50 An embodiment shown in Fig. 27 is provided with, an automatic setting function for the binary 50 
conversion of the luminance variation. In this case, as will be clear when compared with, for 
example, the embodiment of Fig. 4, a picture processing means 181 is so formed that outputs 
of a reference picture memory 181a and input picture memory 181b are provided to an absolute 
difference value circuit 18 1d, an output of the latter of which is provided to a binary circuit 

55 18 1e. An output of a threshold value memory 18 If is provided also to the binary circuit 181e 55 
an output of which is provided to an abnormality discrimination means 182. In the illustrated 
embodiment, the absolute value of a variation corresponding to a difference between the refer- 
ence and input pictures is calculated in the absolute difference value circuit 18 1d. The threshold 
value stored in the memory 1 8 1f is calculated on the basis of "N" input pictures, by- selectively 

60 setting the luminance variation in normal state, with a utilization of the fact that the luminance 60. 
variation in abnormal state is considerably smaller than that in the normal state. 

The threshold value calculation is carried out preferably in accordance with a flowchart of Fig 
28. In this case, it is desirable to set the input picture number N for the threshold value 
calculation to be, for example, 100 and quantity k which will be obtained in accordance with 

65 such formulae as follows to tie 3. In obtaining the quantity k, it is assumed that the luminance 65 
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at a coordinate point P upon receipt of i-th input picture is fip. When the luminance variation 
values in the absence of any abnormality are distributed without any remarkable fluctuation and 
N is sufficiently large, variables //p and op for obtaining S,p are obtained also from the formulae 
as follows: ~ 
5 5 
N N 
/*p=(1/N) 2 fip 2 -/*p 2 , op 2 =(1/N) 2 fip 2 -//p 2 
j=1 i=-1 

10 Here, a following formula is macde to be satisfied, at a probability of (1— <p). 10 

|fp-/zp|<:k<7p 

so as to obtain k, with the luminance of an optional input picture in normal state assumes as fp. 

15 With the N input pictures in normal state provided, the variables HP and <*P are obtained by 15 
means of these formulae, such a reference ppictue that will have the luminance of ^p at a 
coordinate point P, and the threshold value is set to be kcrp obtained by the above operation. 
Then, the probability at which the luminance variation at a point Q where the variation exceeding 
the threshold value has taken place in normal state is known to be <p, while it is possible to 

20 lower tp to a negligible level by optimumly setting k, that is, the probability of erroneous 
operation occurrence can be made less than 1 by setting k to be, for example, 3. It will be 
appreciated that an automatic setting function may be provided for performing the binary conver- 
sion of the luminance variation. Other arrangements and operation of the embodiment of Fig. 27 
are substantially the same as those of the foregoing embkodiments. 

25 In another embodiment shown in Fig. 29, as seen in comparison with the embodiment of Fig. 25 
27, a binary picture memory 191g and a picture processing means 191h are inserted between a 
binary circuit 191e and an abnormality discrimination means 192, so that a binary picture 
storedin the binary picture memory 191g is subjected to a noise processing and the like at the 
picture processing means 19 1h, before being sent to the abnormality discrimination means 192. 

30 While there is a possibility that an output of the binary circuit 191e may have an error of <p, as 30 
has been explained in connection with the embodiment of Fig. 27, this error can be further 
reduced in such manner that, if <p is, for example, an abnormal output caused by the luminance 
variations which occurring at all points within the monitoring zone, a processing of a so-called 
isolated point removal is performed at the picture processing means 19 1h. Other arrangements 

35 and operation of the embodiment of Fig. 29 are substantially the same as those of the foregoing 35 
embodiments. 

Referring to another embodiment as in Fig. 30, the luminance variation in the input picture with 
respect to the reference picture is converted to a binary picture by means of a predetermined 
first threshold value S a at a binary unit 201 e, and this binary picture is labeled at a labeling unit 

40 201f. With respect to every cluster in the labeled pictures, a comparison unit 20 1g counts the 40 
number of objects having an area of more than a predetermined second threshold value S b and M 
compares the counted value with a third threshold value S c . In the case where the counted ^ 
object value exceeds the third threshold value S c , the first threshold value S a for the binary 
conversion is changed so that the binary conversion is carried out again for the labeling. Here, 

45 other picture processing unit 201c corresponds to the picture processing section in the forego- 45 
ing embodiments. According to the present embodiment, therefore, it is made possible to 
exclude, from the objects for providing the abnormality output, such object as rain or snow 
which accompanying a luminance variation continues but of a small luminance difference with 
respect to the background. Other arrangements and operation of the embodiment of Fig. 30 are 

50 substantially the same as those of the foregoing embodiments. 50 
In another emboduiment shown in Fig. 31, as seen in comparison with the embodiment of Fig. 
30, the count at a comparison unit 21 1g exceeding the third threshold value S c will cause the 
second threshold value S b to be changed to provide the same operation as in the embodiment 
of Fig. 30. Other arrangements and operation of the embodiment of Fig. 31 are substantially the 

55 same as those of the foregoing embodiments. 55 
Referring to Fig. 32, there is shown an embodiment which comprises multichannel picture 
input means 220, 220A, . . . 220N respectively connected to each of pairs of reference picture 

memories 221a, 221aA, . . . 221aN and comparison circuits 223, 223A 223N, the latter 

circuits respectively comparing the input picture with the reference picture in respect of their 

60 luminance. Outputs of the comparison circuits 223, 223A, . . . 223N are sent respectively through 60 
an independent line to a common channel selecting control circuit 224 and a common multi- 
plexer 225. In the channel selecting control circuit 224, an incoming luminance variation output 
indicative of an abnormality from, for example, the comparison circuit 223I associated with the I- 
th picture input means 220I causes a select signal ser . a the multiplexer for selecting the l-th 

65 picture input means 220I. The channel selecting control circuit 224 and multiplexer 225 are 65 
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connected to an abnormality monitor unit 226 so that, as soon as the select signal is sent from 
the channel selecting contriol circuit 224 to the multiplexer 225, the multiplexer is made to know 
that the l-th comparison circuit 223I has been selected and to be provided with the luminance 
variation output from the l-th comparison cicuit 223! and passed through the multiplexer 225. 

The abnormality monitor unit 226 comprises such picture processing section of the P'cture & 
input means, abnormality discrimination means and output means as in the foregoing embodi- 
ments and executes the similar picture processing, abnormality discrimination and informing 
operation to those in the foregoing embodiments. The unit 226 also commands the channel 
selecting control circuit 224 to have the select signal for the l-th comparison circuit 22 31 
transmitted contfnuously to the multiplexer 225 until the abnormality discrimination of the picture TO 
from the l-th comparison circuit 2231 is completed, and to have such signal transmission 
terminated upon completion of the abnormality discrimination. 

In the present embodiment, only outputs of the comparison circuits which showing the lumi- 
nance variation are processed, so that any term for which the supervision is disabled can be 
remarkably shortened as compared with the case of a time sharing system which performs the 
supervision with the respective picture input means sequentially switched, and it is made pos- 
sible to effectively prevent any overlooking of abnormal pictures from other picture input means 
than that of which the picture is being processed. 

Referrinq to Fig. 33, there is provided a multichannel supervising system according to an 
20 embodiment of the present invention, which generates no abnormality discrimination output even 
upon occurrence of such pulsating light as lightning or the like. More specifically, picture input 
means 230 230A 230N are provided in multichannel, so that their input pictures are pro- 
vided to a common multiplexer 231, which provides these input pictures through an A/D 
converter 232 to an abnormality monitor unit 233. In the present embodiment, the abnormality 
25 monitor unit 233 comprises the same picture processing means, abnormality discrimination 
means and output means as those in the foregoing embodiments, and performs the same 
picture processing, abnormality discrimination and informing operation also as in the foregoing 
embodiments. The converter 232 is so provided that, upon receipt of an input larger than a 
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242 which receives data from a gain setting memory 243 and operates to make one of the 
analog switches SW, to SW„ through its one output line. Used as the gain setting memory 243 
: . .„ , „,^ki„ momn r» fnr rnrrpsnondence to the picture elements in 1:1 relationship 
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predetermined value, an overflow signal OVF is provided to a gate circuit 234. Also applied to 
30 the gate circuit 234 is a clock signal CLK. When the gate circuit 234 receives the overflow 30 
signal OVF, the gate is turned ON to send clock signals CLK to a counter 235 for the.r counting. 
When the count of the clock signals reaches a predetermined level, the counter 235 provides its 
output to the abnormality monitor unit 233 to have its abnormality discrimination terminated. 
The multiplexer 231 causes the pictures from the picture input means sent sequentially to the 

35 A/D converter 232. __ n 
In the present embodiment, a receipt into at least one of the picture input means 230, 
230A 230N of such pulsating light s lightning causes the luminance variation sent through 
the multiplexer 231 to the A/D converter 232 to be raised to a level higher than a Predeter- 
mined value in the converter, whereby the signal OVF is provided from the A/C converter 232 
40 to the g ate circuit 234, upon which the clock signals CLK are provided to the counter 235 
through the gae circuit 234 so that, when the count at the counter 235 reaches the predeter- 
mined value, the discrimination terminating signal is sent from the counter 235 to the abnormal- 
ity monitor unit 233 to inhibit the informing operation by the output means of the unit 233, tor 
oreventinq any erroneous operation from being caused by such light. 
45 Tn another embodiment shown in Fig. 34. an A/D converter 241 is inserted between a picture 45 
input means 240 and an abnormality monitor unit which includes such picture i processing means, 
abnormality discrimination means and output means as shown in F.gs 32 and 33. The A/D 
converter 242 is provided for an application thereto of a reference voltage V ro( from a plurality of 
reference voltage source V„, V r2 , . . . V (n through analog switches SW, SW 2 . . . . SW„. In the 
illustrated embodiment, the analog switches SW, to SW B are connected to a common decoder 50 



35 



40 



is, for example, a graphic memory for correspondence to the picture elements 
so as to render 512x512 picture elements tobe512x512xm bits. In practice, the m bits are 
55 determined by the number of areas to be set. For example, when 8 areas are set in the 55 
monitoring zone, m is set to be 3. The data in the memory 243 can be provided thereto by 
setting any optional number of the areas with use of a graphic tablet or a light pen. 

Referring to Figs 35 to 36, it is here intended to monitor such a street corner as in Fig. 35 
which including a street lamp RL. by means of the picture input means 240. In this case, such 
60 Trea in the vicinity of the street lamp as enclosed by a dotted line provides a higher luminance 60 
on the monitored picture. When this information on this area is preliminarily provided in the gain 
setting memory 243 as shown in Fig. 36. the gain on such area of the higher luminance can be 
reduced by selectively switching the reference voltage to the A/D converter by means of the 
analog switches receiving a command from the decoder 242 with respect to input picture 
elements of the particular area, so that the entire input picture including the particular area of the 65 
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higher luminance can be monitored with a uniform sensitivity. 

Shown in Fig. 37 is an embodiment in which, as seen in comparison with Fig. 34, a plurality 
of A/D converters 251, 251 A, ... 251 N as well as analog switches SW V SW 2 , . . . SW n respec- 
tively associated with the A/D converters are connected between a picture input means 250 and 
5 an abnormality monitor unit, the switches SW, to SW n being connected to a decoder 252 which 5 
functions, as in the embodiment of Fig. 34, to receive data from a gain setting memory 254 and 
selectively make one of the analog switches through one of output lines of the decoder. 
According to the present embodiment, a digital signal can be switched so that any noise can be 
reduced. Other operation of the embodiment of Fig. 37 is substantially the same as that of the 

10 embodiment of Fig. 34. 10 
In an embodiment shown in Fig. 38, a picture input means 260 is connected to a plurality of 
A/D converters 261a, 261aA,. . .261aN which are coupled respectively independently to each 
of reference voltage sources V r1 , V r2 , . . . V rn and are respectively set to have each of different 
gains. It is assumed here that the l-th A/D converter 261a! has an intermediate gain, i.e., 

15 standard gain and the picture input means receives a normal picture input, then the input picture 15 
is sent through the A/D converter 261al to a subtractor 262 to calculate the luminance variation 
with respect to the reference picture sent from a reference picture memory 261. An output of 
the subtractor 262 is provided to a multiple comparator 263 connected to a plurality of 
reference voltage sources V rs1 , V fs2 , . . . V ren to have "n" threshold values. The comparator 263 

20 determines the extent of the gain modification with respect to the variation output of the 20^ 
subtractor 262 and provides a command to a gain selecting multiplexer 264 connected to the 1 
A/D converters 261a, 261aA, . . . 261aN to select one of the outputs of the converters. The 
gain modification output of the comparator 263 is also provided to another multiplexer 265 for 
modification of reference picture, which multiplexer 265 in turn receives multiplication outputs of 

25 a plurality of multipliers 266, 266A, . . . 266N receiving the output of the reference picture 25 
memory 261. In these multipliers, the reference picture has been multiplied by the same coeffici- 
ents as the mutual gain modification ratios of the A/D converters 261a, 26 1a A, ... 261 aN so 
that, when the gain modification output is provided from the multiple comparator 263 to the 
multiplexer 265, one of the multipliers having the coefficient corresponding to the gain of 

30 selected one of the A/D converters will be selected. The selected gain modification input picture 30 
at the gain selecting multiplexer 264 and the selected multiplied picture of the gain-modification- 
ratio at the reference picture correction multiplexer 265 are both sent to an absolute difference 
value circuit 267 to calculate the absolute value of a difference between these pictures. It will be 
appreciated that an output of the absolute difference value circuit 267 is sent to such binary 

35 circuit, abnormality discrimination means and output means as in Figs. 27 and 29. 35 
When, in the present embodiment, there occurs in the input picture of a monitoring zone an 
abrupt luminance variation due to, for example, a car's headlight, the output of one of the A/D 
converters having a low gain value responsive to light level of the headlight as well as the 
output of one of the multipliers having the same coefficient as the gain modification ratio of the 

40 selected A/D converter are sent respectively through the multiplexers 264 and 265 to the 40 
absolute difference value circuit 267 to calculate the absolute value of the difference for process- | 
ing the picture at the subsequent stage. That is, when the gain of the A/D. converter is selected 1 
to be 0.8 multiplication, gain correction of 0.8 is realized also with respect to the reference 
picture. Therefore, even the abruptly increased or decreased luminance of the input picture will 

45 cause the gain correspondingly increased or decreased, so that the monitoring can be carried out 45 
always with a uniform sensitivity over the entire input picture. 

Shown in Fig. 39 is an embodiment which realizes the abnormality supervision by means of 
two-dimensional displacement vector. More specifically, an input picture of a picture input means 
270 is converted to a binary picture at a binary circuit 271 and then sent to an area measuring 

50 circuit 272 which counts the number of picture elementsin the binary picture having a value of 50 
"1" to determine the area AR, of a monitoring object and sends the area to an area ratio 
calculating cicuit 273. In the illustrated embodiment, the area ratio calculating circuit 273 is 
holding an area AR 0 of the binary picture obtained from a previous input picture of the picture 
input means 270, so that a variation ratio between the previous picture area AR 0 and the input 

55 picture area AR„ that is, AR=| AR,-ARo|/AR 0 is calculated in the circuit 273, and this calculated 55 
area ratio is sent to a vertical displacement calculating circuit 274 to determine a vertical 
displacement AX in accordance with an equation 



60 



AX = A.sgn(AR t — AR 0 ) SQRT(AAR) 

60 

wherein the term SQRT(AAR) is the square root of AAR. The area variation is proportional to the 
square of a displacement seen in the object and, so long as the actual area of the object is 
assumed to be substantially constant, the area variation is proportional to a vertical displacement 
of the object. The term sgntAR, — AR 0 ) is a sign function which has a value of +1 when 
65 (AR,-AR 0 ) is of a positive value or zero, or a value of -1 when (AR,-AR 0 ) is a negative value, 65 
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whereby the vertical displacement of AX is made to have a positive value when the object 
approaches the picture input means but to have a negative value when the object separates the 
picture input means 270. Further, the term A is a coefficient for conversion into the actual 
displacement of the object. 
5 On the otherhand, the binary picture is also provided to a horizontal displacement calculating 
circuit 275 which determines the central position of the binary picture, as well as a difference 
between a horizontal position Y, of the input picture and a horizontal position Y 0 of the previous 
binary picture held at the circuit 275, that is, AY = Y,-Y 0 is obtained here. This output of the 
horizontal displacement calculating circuit 275 is provided, together with the above output of the 
10 vertical displacement calculating circuit 274, to a two-dimensional displacement vector output 
circuit 276 so that, even when the object approaches the picture input means 270 as shown in 
Fiq 40, the output circuit 276 provides a displacement vector. 

Accordingly, such a displacement vector (AX, AY) as shown in Fig. 41c can be calculated on 
the basis of such previous binary picture as shown in Fig. 41a and such latest binary picture as 
15 shown in Fig 41b. While, in a system which calculaes the displacing distance without using the 
area ratio for calculating the vertical position but only on the basis of the central position of the 
binary picture similarly to the horizontal position calculation as shown in, for example. Figs. 42a 
and 42b and which employs for monitoring the moving object in particular such TV camera 
directed 'obliquely downward as shown in Fig. 40, the displacing distance of the object is 
20 moving at a constant speed, such displacing distance can be measured always accurately 2U 
according to the present embodiment. It should be understood that the displacement vector 
calculated in the above manner can be converted to a speed vector by dividing respective 
displacement vector components by the measuring time interval. The arrangement of the present 
embodiment may be effectively employed, for example, as a part of the picture processing 
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25 means of Fig. 1. 

In Fig 43 there is shown an embodiment in which, as is clear in comparison witn tne 
embodiment of Fig. 1, a picture input means 280 includes such a picture pickup means as a 
color TV camera and sends three primary-color signals of red, green and blue to a color tone 
extracting means 281 which extracts hues for allowing such expressions as G/R, R/<R+G-l-B) 
30 and G/(R+G + B) possible and calculates the number of picture elements indicative of colors 3U 
themselves not depending on the light level. . ... 

An output of the color tone extracting means 28 1 is sent to an abnormality monitor unit 283 
including for example, such picture processing means, abnormality discrimination means and 
output means as shown in the foregoing embodiment of Fig. 27 to execute therein and same 

35 picture processing, abnormality discrimination and informing operation as those of the foregoing 3b 
embodiment. As will be seen in comparison of Figs. 44a and 44b showing an example of the 
input picture through the hue extraction in the present embodiment with Figs. 45a and 45b 
showing an example of monochromatic input picture according to the foregoing embodiment, it 
becomes impossible in the monochromatic input picture to monitor an object which enters into a 

40 shade of a building due to daylight since the luminance variation is very slight in the shade area. 4U 
According to the present embodiment, however, the number of picture elements indicative of 
colors themselves is processed so that such building shade will not appear in the input picture 
and luminance contrast will be kept substantially constant, whereby the object monitoring is 
made reliable. In addition, the arrangement of the present embodiment is effectively used even 

45 when the monitoring zone includes scattering areas which are illuminated and not-illuminated. 4b 
In an embodiment shown in Fig. 46, as seen in comparison with, for example, the embodi- 
ment of Fig 4 a texture operating means 299 and an automatic detection-area setting means 
300 are inserted between a reference picture memory 291b of a picture processing means 291 
and a detection area memory 297. In the illustrated embodiment, the texture operating means 

50 299 is provided with means for receiving an input picture through the reference picture memory 50 
291b and calculating the power spectrum of the picture to obtain the texture feature values and 
the power spectrum is calculated for each of very small areas within the monitoring zone. The 
automatic detection-area setting means 300 preliminarily registers therein, as the texture feature 
values such power spectrum patterns as a fence, concrete wall, trees, ground surface, sky and 

55 the like so that, when the monitoring zone is as shown in, for example. Fig. 47, the automatic bb 
detection-area setting means 300 compares the power spectrum patterns from the texture 
operating means 299 with the registered reference patterns to discriminate correspondence of 
the very small area of the calculated power spectrum to such particular objects as the fence, 
trees and the like, and automatically provides the data of the warning levels to the detection ^ 

60 area memory 297. _ . ... u " 

The operation of the present embodiment will be explained with reference to Fig . 48, in which 
diagrams (a), (c) and (e) are showing horizontal or X-directional power spectra and diagrams (b), 
(d) and (f) are showing vertical or Y-directional power spectra, where the horizontal and vertical 
axes represent frequency f and the power |FX|* or |FY|* of the frequency components. The 

65 diagrams (a) and (b). (c) and (d), and (e) and <f) show the power spectra of the very small areas 65 



BNSDOCID: <GB 2183876A_J_> 



13 GB2 183878A 13 

of such an object having a small luminance variation as the concrete wall or the ground, of such 
an object involving the shake as the tree, and of such an object having many vertically extending 
members as the fence, respectively. On the basis of these data, the supervising ability for that 
monitoring zone of Fig. 47 can be enhanced in such that, for example, the small area judgeable 
5 to be the tree in view of the data of the diagrams (c) and (d) of Fig. 48 is preliminarily set to 5 
have a low or zero warning level, the small area judgeable to be the conrete wall or ground in 
view of the data of the diagrams (a) and (b) of Fig. 48 is set to have a warning level 1 , and 
further the small area judgeable to be the fence because of such power which is high only in the 
vertical direction as in the diagrams (e) and (f) of Fig. 48 and appering to be easy to intrude 

10 therethrough is set to have a warning level 2, as indicated in Fig. 47. 10 
Other arrangements and operation of the present embodiment of Fig. 46 are substantially the 
same as those of the embodiment of Fig. 4, except that the information is provided to the 
detection area memory 297 in the above-mentioned manner. In Fig. 47, = onstituent members 
corresponding to those in the embodiment of Fig. 4 are denoted by the same reference 

15 numerals but added by 270. 15 
Shown in Fig. 49 is another embodiment of the abnormality supervising system according to 

the present invention, in which a plurality of picture input means 310, 3 OA 310N are 

operatively associated respectively with each of such picture processing means as that in the 
foregoing embodiments, and difference circuits 311c, 311 cA 311cN included in the picture 

20 processing means calculate the luminance variations between the latest input pictures and the 20^ 
reference pictures of reference picture memories 311b, 31 1bA, ... 31 1bN, results of which ^ 
calculation are sent to a moving object identifying means 312. This identifying means 312 can 
process the N input picture signals for an operation of obtaining a wide range moving locus of 
the object, and also can take either object monitoring or area setting mode. Also operatively 

25 coupled to the identifying means 312 is an overlap portion setting means 313 which includes a 25 
monitor video 314 and light pen 315 forming a pointing means for specifying set positions on 
video screen, as well as a picture memory 316 for storing the set positions. 

Referring to the operation of the present embodiment with reference to Figs. 50 to 52, it is 
assumed here that, upon installation of the present system, TV cameras 317 and 317A forming 

30 the picture pickup means of the picture input means 310 and 31 OA are positioned to monitor a 30 
zone of a passage within a building in opposing direction as shown in Fig. 50, so that the 
camera 317 provides a picture of Fig. 51 and the other camera 317A provides a picture of Fig. 
52. Now, the moving object identifying means 312 is placed in the area setting mode, an 
overlap portion between the monitoring zones of the both cameras 317 and 317A is divided 

35 into, for example, such twelve closed areas as shown in Figs. 51 and 52, preferably, by drawing 35 
them on the screen of the monitor video 314 with the light pen 315, the closed areas are 
stored in the picture memory 316 and also superimposed on the screen of the monitor video 
314 for an operator's confirmation. Then the moving object identifying means 312 is placed in 
the object monitoring mode. If an object moves as shown by an arrow in Fig. 51 or 52 and 

40 enters into one of the closed areas designated by 9 in the monitoring mode, the object is 40 
located at the same closed area 9 in the overlap portion of the monitoring zones of the two m 
cameras 317 and 317A and thus the identifying means 312 can easily identify that the object is 1 
identical to each other. 

Therefore, in the present embodiment, a wide range monitoring zone can be set with a 

45 plurality of the picture input means 310, 31 OA 31 ON while allowing them to define a 45 

common overlapping portion of respective monitoring zones of them, whereby-any wide range 
movement of the object can be enabled to be effectively tracked. In addition," the present 
embodiment can be effectively utilized as incorporated into the abnormality discrimination means 
of, for example, Fig. 1. 

50 Referring to Fig. 53, there is shown an embodiment in which, as is clear in comparison with bO 
the embodiments of Figs. 1 and 4, a labeled output is provided from a picture processing unit 
321c which receives outputs of a picture input means 321, to a labeling picture memory 326, 
and an output of this memory 326 and the picture signal from an input picture memory 321a 
are both provided to an operating circuit 327, an output of which is provided to a reference 

55 picture memory 321b. A reference picture signal from the reference picture memory 321b is 55 
sent, as in the foregoing embodiments, to the picture processing unit 321c to calculate the 
luminance variation between the latest input picture signal from the picture input means 320 and 
the reference picture signal from the memory 321b. In the present case, the picture processing 
unit 321c sends, to the labeling picture memory 326, an output at such labeling step immedi- 

60 ately before such extracting step as in the picture processing algorithm of Fig. 2. When, on the 60 
other hand, the operating circuit 327 receives a binary output of "0" from the labeling picture 
memory 326, i.e., when there is no luminance variation, the operating circuit 327 provides the 
input picture signal as it is to the reference picture memory 321b, whereas, upon receipt of a 
binary output of "1" from the memory 326, i.e, when there is a luminance variation, the circuit 

65 327 stops the transfer of that portion in the input picture signal of the luminance variation, and 65 
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an area of this variation is masked. 

Referring to Fig. 54, when such an input picture including an object as in Fig. 54a is present, 
an area of the object is labeled with "1" and the other area is labeled with "0" in the labeling 
step as shown in Fig. 54b, in the picture processing unit 321c, and the area having the binary 
5 value of "1" is masked in the operating circuit 327. As a result, as shown in Fig. 54c, a 5 
reference picture that has an unrenewed area corresponding to the object and enclosed by a 
dotted line within other renewed area is sent from the reference picturememory 321b to the 
picture processing unit 321c. In this way, the present embodiment can improve the reliability of 
the reference picture. Other arrangements and operation of the embodiment of Fig. 53 are 

10 substantially the same as those of the foregoing embodiments. 10 
Shown in Fig. 55 is another embodiment, wherein outputs of a plurality of sensors 330, 
330A, . . . 330N are provided to an abnormality discrimination means 332 which includes a 
deduction means 334 for discriminating the absence or presence of an abnormality on the basis 
of information from a knowledge base 333. These sensors are properly arranged in a monitoring 

1 5 zone to suitably combine information detected by the sensors on the basis of the information 1 5 

from the knowledge base 333 for abnormality discrimination. In the case where first, second and 
third groups of the sensors are installed, for example, in the vicinity of a concrete wall, an outer 
house wall and a house entrance of a residential ground, respectively, and it may be possible to 

• discriminate a presence of an intruder when there is a continuous detection of outputs from the 
20 first to third sensor groups in the nighttime. 

A relatively simpler example of the embodiment of Fig. 55 is shown in Fig. 56, which uses a 
first infrared-ray sensor 340 of two opposing elements installed in both sides of an entrance 
gate of a residential ground, a second reflection-type-ultrasonic, electric-field detection or the like 
type sensor 340A installed in the vicinity of a house window, and a third pane-break sensor 
25 340B installed on a pane of the same window. With these sensors, the abnormality information 25 
can be sequentially sent from such sensors to an abnormality discrimination means 342 and, if 
necessary, the discriminated abnormality can be stepwise informed by an output means 345. 

The embodiments of Figs. 55 and 56 can be incorporated in the arrangements of the forego- 
ing embodiments to contribute to the expansion of the expert system as well as to the 
30 improvement in the reliability. 30 
In another embodiment shown in Fig. 57, as is clear in comparison with the embodiment of 
Fig. 1, an output of a°picture processing means 351 is provided to a mask picture producing 
means 356 an output of which is provided to a mask picture memory 357 to be stored therein 
for use in one of the steps in the picture processing algorithm of the processing means 351. 
35 When, for example, a tree located within the monitoring zone is shaken to cause a luminance 35 
variation to be likely to cause an erroneous operation of the system, the mask picture producing 
means 356 masks the tree in the input picture. Since this enables it possible to ignore any 
luminance variation at an area likely to cause the erroneous operation and thus to follow the 
picture processing algorithm, a highly reliable supervision can be effected. 

• 40 A practical example of the embodiment of Fig. 57 is shown in Fig. 58, in which a picture 40 
processing means 361c practically may have the same arrangement as, for example, the picture 
processing means in the embodiment of Fig. 27, and the same constituents as those in Fig. 27 
are denoted by the same reference numerals but added by 180. In the present instance, a mask 
picture producing means 366 comprises an integral circuit 366a receiving an output of an 
45 absolute difference value circuit 36 1d of the picture processing means, and a binary circuit 366b 45 
receiving an output of the integral circuit and a predetermined threshold value, and this integral 
circuit 366a functions to add a predetermined number of input pictures, so that the integral 
circuit 366a provides data including a relatively larger integration value for an area where the 
luminance variation occurs frequently in the picture of the monitoring zone, as well as a relatively 
50 smaller integration value for other area in the picture. Such data are converted at the binary^ 50 
circuit 366b with a threshold value into a binary mask picture in which a binary value of "1" is 
given to the area of the frequent luminance variation and a binary value of "0" is given to the 
other area. The mask picture is sent from a mask picture memory 367 again to the picture 
processing unit 361c of the picture processing means so that, when such an input picture of the 
55 monitoring zone as shown in Fig. 59 is being obtained, an area including a tree in the monitoring 55 
zone as enclosed by a dotted line is treated as a masked area MSK any luminance variation 
occurring in which is to be ignored in performing the abnormality discrimination. Other arrange- 
ments and operation of the present embodiment are substantially the same as those of the 
foregoing embodiments. cr\ 
60 In the example of Fig. 58, the input to the integral circuit 366a of the mask picture producing bO 
circuit 366 is obtained from the absolute difference value circuit 36 1d of the picture processing 
means. However, as seen in Fig. 60, the same operation as in Fig. 58 is attainable even when 
an input to a mask picture producing means 376 is obtained from a binary circuit 37 1e at a 
subsequent stage of an absolute difference value circuit 37 1d in a picture processing means. 
65 Shown in Fig. 61 is another embodiment in which, as is clear in comparison with the 65 
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embodiment of Fig. 4, a plurality of detection area memories 387, 387A, . . . 387N are arranged 
between an area setting means 386 and an abnormality discrimination means 382. In the 
present embodiment, different detecting sections of such a relatively broader monitoring zone as 
a factory site are stored as detection objects and the abnormality discrimination is executed in 
5 different modes for the respective detecting sections. In this case, the areas to be stored in the b 
detection area memories should be such sections in the site as a place in the vicinity of an 
entrance door of the factory site, a place where such fire-involving equipment as welding 
machine or the like is used, areas in which industrial robots, operatorless carrier vehicles or the 
like are in operation, and so on. According to the present embodiment, therefore, the monitoring 
10 of such different sections in the zone can be achieved by commonly using a picture input means iu 
380 picture processing means 381a. 381b and 381c, the major part of the abnormality 
discrimination means 382 and an output means 385, and such different sorts of monitoring as 
intruder monitoring, fire preventive monitoring, production line monitoring and so on can be 
realized with use of a single abnormality monitoring unit. Other arrangements and operation of 
15 the embodiment of Fig. 61 are substantially the same as those of the foregoing embodiments. lb 
Referring to Fig. 62, there is shown another embodiment, in which, as is seen in comparison 
with the embodiment of Fig. 1, a detection area transfer means 396 is provided for receiving an 
output of a picture processing means 391 to transfer the detection area in response to a 
displacement of the object and to provide an output to an abnormality discrimination means 
20 392, so as to concentrate the monitoring function on the moving object. 2U 
Shown in Fig. 63 is a more practical example of the embodiment of Fig. 62, in which a 
detection area transfer means comprises an object extracting unit 406 receiving an output of a 
picture processing means 401, a coordinate converting unit 407 receiving an output of the 
extracting unit 406 and a memory 408 providing the data of the detection object area to the 
25 coordinate converting unit 407. In the object extracting unit 406, such picture processing as 25 
shown in Fig. 2 is carried out in the picture processing means 401 in such that the most similar 
object is extracted by means of the pattern matching operation or the like from the feature 
values of the object obtained at the feature-value operating step, and the center coordinates of 
the extracted object are calculated. The coordinate converting unit 407 shifts a detection area P 
30 enclosing the center C of the object as shown in Fig. 64 and stored in the detection area 
memory 408. so as to follow the movement of the object on the basis of the center coordi- 
nates of the object obtained at the object extracting unit 406. That is, when the object located 
at a position Ka on the lower left side of the picture as shown in Fig. 65a moves to a position 
Kb on the upper right side of the picture as shown in Fig. 65b, the coordinate converting unit 
35 407 shifts the respective detection areas from Pa to Pb, following the movement of the object 
Provided that the object is present always in the input picture, or in other words, where the 
monitoring zione is set to include an area in which the object is moving, then such a picture in 
which the object is absent will be required as the reference picture for the picture processing 
means In this event, such reference picture may be obtained in such that, when, for example, 
40 such an object OBJ as an operatorless vehicle reciprocating along a rail RAL is located at the 
lower left side of an input picture as shown in Fig. 66a and the object OBJ is located at the 
upper right side in another input picture as shown in Fig. 66b, these input pictures are com- 
posed into a picture of Fig. 66c. 
A more practical example of the embodiment of Fig. 62 may be of such an arrangement as 
45 shown in Fig. 67. In the present instance, as seen in comparison with the embodiment of Fig. 
63 a coordinate converting unit comprises subtraction circuits 417 and 41 7A connected in 
parallel to an abnormality discrimination means 412, an object extracting unit 416 and a detec- 
tion area memory 418. Center coordinates X, and Y, of an object are sent from the object 
extracting unit 416 to both subtraction circuits 417 and 417A, while coordinaes X 2 and Y 3 of a 
50 luminance variation are sent from the abnormality discrimination means 412 also to the subtrac- bu 
tion circuits 417 and 41 7A for operations therein of X 3 =X 2 -X, and Y 3 =Y 2 -Y„ and thereby 
address coordinates X 3 and Y 3 are provided to the detection area memory 418 for its accessing, 
so that the detection area is transferred following the movement of the object. 
In the foregoing detection area transferring means, it is preferable to set, as shown in Fig. 68 
55 and in addition to the detection area P, a tracking area Q made to be of the maximum moving 55 
range of the object, for example, during one sampling time, with respect to the object having a 
center C. This enables the concentrated monitoring only for the tracking area Q and speeds up 
the abnormality discrimination. Other arrangements and operation of the embodiments of Figs. 
62 to 68 are substantially the same as those of the foregoing embodiments. 
60 In an embodiment shown in Fig. 69, as is clear in comparison with the embodiment of Fig. 4, 
the discrimination means comprises a main abnormality discriminatin means 422a and a sub- 
abnormality discrimination means 422b, and the detection area memory comprises a main 
detection memory 427a and a plurality of sub-detection area memories 427b, . . . 427blM. An 
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output of the main detection memory 427a is provided to the mam abnormality discrimination 
65 means 422a, and outputs of the sub-detection memories 427b, . . . 427bN are provided to the 
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sub-abnormality discrimination means 422b. In this case, the main detection area memory 427a 
stores set areas of a monitoring zone for a relatively rough discrimination, while the sub- 
detection area memories 427b, . . . 427bN store set areas of the monitoring zone for a relatively 
minute discrimination. In the present embodiment, the rough discrimination is first made and, if 
5 an abnormality is detected by this rough discrimination, then the minute discrimination is further 5 
made, whereby the informing operation of the supervising system is made more highly reliable. 
Other arrangements and operation of the present embodiment are substantially the same as 
those of the foregoing embodiments. 

Referring to Fig. 70, there is shown another embodiment in which, as seen in comparison with 

10 the embodiment of Fig. 1, the picture processing mens provided between a picture input means 10 
430 and an abnormality, in particular, intruder discrimination means 432 includes an object 
extracting means and an object tracking means. In the illustrated embodiment, the object extract- 
ing means includes an input picture memory 431a and a reference picture memory 431b both 
receiving an output of the picture input means 430, as well as an object extracting unit 436 

15 receiving outputs of the both memories, while the object tracking means includes an extracted 15 
input object picture memory 437, a extracted former object picture memory 437A and an object 
tracking means 438 receiving outputs of the both memories and, if necessary, an output of an 
attribute memory 439. 

In the object extracting means 436, the same picture processing as that of the picture 

20 processing means 21c in the embodiment of, for example, Fig. 4 is performed, i.e., an input 20 
picture is subjected to the binary conversion and labeling. The labeled binary picture is sent to 
the extracting memories 437 and 437A. In an event where the input picture sent from the 
extracting means 436 is shown in Fig. 71 and stored in the extracted input object picture 
memory 437, while such a picture as shown in Fig. 72 is previously stored in the extracted 

25 former object picture memory 437A upon receipt of previous input picture, objects designated 
by figures 1 to 5 are known to have been moved, and a tracking of the objects is to be 
effected for identification of them in the object tracking unit 438. 

More in detail, the identifying operation at the object tracking unit 438 is earned out such that, 
if an object OBJ A in the latest input picture and an object 0BJ p in the previous picture Partly 

30 overlap to form an overlapping portion shown as hatched in Fig. 73, these objects are judged to 
be an identical object. On the other hand, when the sampling speed of the picture of the system 
is lower than the moving speed of the object and there is no overlapping portion between the 
objects in the input and previous pictures, the system predicts a position of the object upon 
extraction of the latest input picture on the basis of a displacement vector of the object OBJ 

35 upon extraction of the previous picture to obtain such a predictive object OBJ p ' as shown in Fig. 35 
74 and judges an object having such a hatched portion overlapping with the predictive object 
OBJ ' on the input picture to be identical. When a predictive object is obtained as overlapping 
with both objects OBJ1 and OBJ2 on the input and previous pictures as shown in Fig. 75 and 
thus it is impossible to identify the objects, the object identifying operation can be achieved by 

40 finding such shape parameters of the both objects as their sizes, major axis ratio, etc., and 40 
discriminating them on the basis of the similarity. 

Further, in an event where information of a tree or the like located in the monitoring zone to 
allow an object to be hidden behind it is previously stored in the attribute memory 439 
operatively associated with the object tracking unit 438, the identity discrimination of such 

45 object the luminance variation due to which is caused to temporatily disappear when coming 4b 
behind the tree can be still performed when the luminance variation again takes place in the 
vicinity of the tree. 

In this way, according to the embodiment of Fig. 70, a continuous object tracking can be 
executed and an accurate abnormality discrimination can be realized. Other arrangements and 
50 operation of the present embodiment are substantially the same as those of the foregoing 50 

embodiments. . . . 

Fig 76 shows a means for automatically correcting the diaphragm of the TV camera included 
in the picture input means in the foregoing embodiments. The automatic diaphragm correcting 
means includes a signal detecting means 446 which receives an output of a picture input means 

55 440 and an output of a detection area setting means 447, an output of the signal detecting 55 
means 446 being provided to a diaphragm correcting means 448 which in turn provides a 
diaphragm correction signal as its output to the picture input means 440. Now, when the picture 
input means 440 provides such a picture as shown in Fig. 77, such a detection area as 
enclosed by a dotted line in the picture is set by the detection area setting means 447, and the 

60 picture is processed by the signal detecting means 446 and diaphragm correcting means 448 60 
without being subjected to the luminance variation of the other area in the picture than the 
detection area, and is then subjected to the picture processing at the subsequent stage. 

As the signal detecting means in the embodiment of Fig. 76, optimumly, a peak value 
detecting means 456 is employed as shown in Fig. 78, in which the means 456 is provided 

65 with a peak-value hold circuit which receives the peak value, i.e., the maximum luminance level 65 
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of a picture signal received through, for example, an analog switch, and a diaphragm correction 
signal is generated through a diaphragm correcting means 458 according to the maximum 
luminance level, and to send it to a picture input means 450. The signal detecting means in Fig. 
76 may also comprise, as shown in Fig. 79, an integral value detecting means 466 which 
5 integrates the luminance level of input pictures to obtain an average luminance level, to generate 5 
the diaphragm correction signal through a diaphragm correcting means 468 and to send it to a 
picture input means 460. 

Further it will be appreciated that the area setting at the detection area setting means 44/, 
457 and 467 can be carried out by means of a graphic tablet or the like. The output of each of 

10 the picture input means 440, 450 and 460 subjected to the diaphragm correction is used for the 10 
picture processing, abnormality discrimination and informing operation, as already explained in 
connection with the foregoing embodiments. 

In another embodiment shown in Fig. 80, the picture input means is formed so that an output 
of a reference picture memory 471a is provided to an absolute difference value circuit 47 1d 

15 together with an input picture signal, and also through a multiplier 471b for multiplication of a 15 
constant K smaller than 1 to a threshold value picture memory 471c which uses as a threshold 
value the output of the multiplier 471b. Outputs of the threshold value picture memory 471c and 
absolute difference value circuit 47 1d are both sent to a comparison circuit 47 1e to convert the 
input and reference picture signals into binary picture signals for the picture processing at the ^ 

20 subsequent stage. ^ 
The operation of the present embodiment will be explained with reference to Fig. 81. In the ^| 
drawing, a solid-line signal waveform corresponds to one horizontal line in the input picture, M 
and N are ranges corresponding to parts of the horizontal line having high and low luminance, 
respectively, and relatively high and low peak values P1 and P2 of the signal indicate objects in 
25 the high and low luminance ranges. When the threshold value provided to the comparison circuit 25 
47 1e is constant, as shown by dotted line curves in the drawing, the signal waveform becomes 
constant in its vertical width regardless of the magnitude of the luminance, i.e., brightness and 
darkness of the picture, so that there is a risk that even an object presnt in the range N does 
not cause the signal to reach the threshold level and thus no abnormality can be discriminated. 

30 According to the present embodiment, on the other hand, a reference pictue without any 30 
abnormality is multiplied by the constant K smaller than 1, for example, 0.3, to provide a 
variable threshold value, and this value is sent through the threshold value picture memory 47 1e 
to the comparison circuit 47 1e, so that, as shown by chain-line curves in Fig. 81, the threshold 
curve varies largely in its vertical width in the range M but slightly in the range N depending on 

35 the brightness and darkness of the picture. Therefore, the object can be positively grasped and 35 
the system can be remarkably improved in the reliability. 

Further, as shown in Fig. 82, the threshold value picture memory in Fig. 80 may be replaced 
by a latch circuit 481c which is connected in parallel with another latch circuit 481cA provided 
between an absolute difference value circuit 48 1d and a comparison circuit 48 1e, to provide a 

40 variable threshold value and a luminance variation signal simultaneously to the comparison circuit 40 
48 1e through the both latch circuits 481c and 481cA, whereby the same operation as in the A 
embodiment of Fig. 80 can be attained. In the embodiments of Figs. 80 to 82, the picture input ^ 
means, other part of the picture processing means, abnormality discrimination means and output 
means have substantially the same arrangements as those in the foregoing embodiments. 

45 Referring to Fig. 83, there is shown an embodiment in which, as seen in comparison with the 4b 
embodiment of Fig. 4, a picture processing means provided between a picture input means 490 
and an abnormality discrimination means 492 includes a first subtractor 49 1d which receives 
outputs of an input picture memory 491a and a first reference picture memory 491b and 
performs a subtraction over them, to function to remove stable background in the picture of the 

50 monitoring zone, an output of the subractor 49 1d being sent to a second subractor 49 1e and 5U 
then to an abnormality processing means 491c. The second subtractor 49 1e also receives an 
output of a second reference picture memory 491bA which in turn receives an output of the 
picture processing means 491c through a multiplier 49 1f. 

The operation of the present embodiment will be explained with reference to Figs. 84 and 8b. 

55 When such an input picture including an abnormal object OBJ2 and shown in Fig. 84b appears 55 
with respect to such a reference picture including normal object 0BJ1 as shown in Fig. 84a, 
such a subtraction picture including only the abnormal object 0BJ2 as shown in Fig. 84c is 
obtained from the first-stage subtractor 49 1d. In this case, the picture processing part 491c at 
the subsequent stage converts its input to a binary signal according to such a predetermined 

60 threshold value V T „ as shown in Fig. 85 but, when the output of the subtractor 49 1e contains 60 
an impulsive noise N, produces an output B restrained to be lower than the threshold level V TH 
through a filtering to have the picture vignetted, whereby the transfer of the abnormal picture 
signal due to such noise N is restrained and the impulsive noise can be eliminated. In this 
manner, it is made possible to eliminate any minor noise in such case where the background is 

65 stable or stationary in theinput picture as a monitoring of a room interior. 65 
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Apart from such impulsive noise as above, a shake or the like occurring in the normal object 
OBJ1 in the input picture of, for example, an outdoor monitoring zone may cause an abnormality 
output to be provided even in the absence of abnormality, due to that the luminance variation 
takes place at a position corresponding to the object OBJ1 as shown in Figs. 84d and 84e. In 
5 the present embodiment, the fact that the luminance variation due to the shake of tree or the 5 
like object 0BJ1 takes place at the same position is taken into consideration, and an input 
picture immediately before the latest input picture is stored in the second reference picture 
memory 491bA so that any luminance variation due to such shake or the like is subjected to the 
subtraction at the second subtractor 49 1e to thereby eliminate the minor noise. In storing the 

10 "immediately-before" input picture in the second reference memory 491bA, it should be avoided 10 
that even a picture involving an abnormality in practice is stored as a reference picture by 
making the "immediaely-before" input picture as it is to be the reference picture. For this 
nurpose, the input picture having any luminance variatio is multiplied by a constant C smaller 
than 1 for example, 0.5 at the multiplier 49 1f, and the thus multiplied picture is stored in the 

15 second reference picture memory 491bA as the reference picture, whereby the luminance vana- 15 
tion of the minor noise can be reduced to be half. Since the variation on the input picture is 
small enough in the area, the variation can be effectively removed at the subsequent stage 
picture processing means 491c for the filtering and binary conversion. Accordingly, the transfer 
of the abnormality output caused by the shake of the background object can be prevented and 

20 the system can be improved in the reliability. Other arrangements and operation of the embodi- 
ment of Fig. 83 are substantially the same as those of the foregoing embodiments. 

Referring to Fig 86, there is shown an embodiment in which, as seen in comparison with Fig. 
1 a picture processing signal and a detection signal of an external sensor 516 are provided to 
an abnormality discrimination means 512 to realize an expansion of the expert system. As the 

25 external sensor 516, a sensor providing a distance to an object, a temperature sensor or any 25 
other sort of sensor may be utilized. 

Fiqs 87 and 88 are showing the whole conceptional appearance of the present system and 
the information processing steps of the system, respectively. From these drawings, the manner 
in which the respective embodiments disclosed are practiced will be readily understood Various 

30 installation examples of the present system are shown in Figs. 89 to 96, wherein the different 30 
warning levels are numerically given as an example in each monitoring picture of the respective 
drawings From these examples, it should be appreciated that the system according to the 
present invention is so high in the wide adaptability of its use that, as seen in Fig. 96, for 
example, the system can be employed for informing a danger when an infant playing in a room 

35 approaches a staircase, bathroom or like place, and so on. 



1 An abnormality supervising system comprising a picture input means for monitoring a zone 
to be supervised, a picture processing means for comparing an input picture obtained from said 

A 40 picture input means with a reference picture stored and processing said input picture to obtain 40 
information necessary for an abnormality discrimination, an abnormality discrimination means 
including memory means for preliminarily storing reference information necessary for said abnor- 
mality discrimination and to be compared with said information obtained from said picture 
processing means to discriminate an abnormality of an object in said monitoring zone, and an 
45 output means receiving an output of said abnormality discrimination means. 45 

2 A system according to claim 1 , which further comprises a detection area setting means 
providing an output to said abnormality discrimination means, said output being of divided 
detection areas in said input picture and having respectively different warning levels. 

3 A system according to claim 2, wherein said warning levels are changeable. 

50 4-' A system according to claim 2, which further comprises a variation pattern memory means 50 
for storing a luminance variation pattern in abnormal state, an output of said variation pattern 
memory means being provided to said abnormality discrimination means. 

5 A system according to claim 4, wherein said variation pattern is set to grasp said object 
moving from one of said areas which is low in said warning level to another area high in the 

55 d "n level 

warning ev^^ according to dajm 2 , wherein said detection areas are settable on said input 

picture monitored in any optional shape. 

7 A system according to claim 2, which further comprises means for providing to said input 
picture an attribute area in addition to said detection areas and storing a characteristic corre- 

60 sponding to said attribute area, an attribute output of said attribute area providing means being 60 
provided to said abnormality discrimination means. 

8 A system according to claim 2, wherein a plurality of said detection area setting means 
are provided for monitoring respectively each of a plurality of sections of said monitoring zone 
which is relatively broad. CK 

65 9. A system according to claim 8, wherein said detection area setting means is provided for bt> 
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automatically setting said monitoring sections. _ 

10 A system according to claim 1. which further comprises means for masking an area in 
said input picture having a luminance variation discriminated not to be abnormal, an output of 
said masking means being provided to said picture processing means. 
5 1 1 . A system according to claim 2, which further comprises means for shifting said detection 
areas as said object moves within said monitoring zone. 

12 A system according to claim 11, wherein said detection area shifting means includes 
means for extracting said object from said input picture, a memory for storing a detection area 
set in the picture to enclose the object, and a coordinate conversion means for causing said 

10 detection area to be moved to keep enclosing the object moved. 

13 A system according to claim 1, which further comprises means for discriminating a 
normal variation pattern from an abnormal variation pattern of said input picture. 

14 A system according to claim 1, which further comprises means for extracting said object 
from said input picture, and means for tracking movement of said extracted object. 

15 15 A system according to claim 14, wherein said object tracking means includes means for 15 
predicting a moving position of said object, and means for identifying the object. 

16 A system according to claim 1, wherein said abnormality discrimination means discrimi- 
nates said abnormality on the basis of a moving locus with time of said object. 

17 A system according to claim 1, wherein said picture input means includes a plurality of 
20 picture pickup means installed for overlapping their monitoring zones on each other. 

18 A system according to claim 1, wherein said picture processing means comprises means 
for inhibiting renewal of said reference picture when said input picture has a luminance variation. 

19 A system according to claim 18, wherein said reference picture is obtained by averaging 
siqnals of a plurality of said input pictures which are discriminated normal. 

25 20 A system according to claim 19, wherein said input picture except any part thereof 1* 
showing a variation keeps renewing said reference picture in said picture processing means. 

21. A system according to claim 1, wherein said reference picture is provided in a plurality in 
said picture processing means. 

22. A system according to claim 21, which further comprises means for preparing another 
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30 reference picture for eliminating minor noise. 

23. A system according to claim 1, wherein said abnormality discrimination means receives 
an output of an external sensor together with said input picture. 

24 A system according to claim 23, wherein said external sensor is a distance sensor. 
25' A system according to claim 23, wherein said external s-.sor is a temperature sensor. 
35 26' A system according to claim 23, wherein said external sensor is a sensor for detecting 
an object located at a dead angle position of said monitoring zone in said input picture. 

27 A system according to claim 1, wherein said picture processing means includes for 
automatically setting a threshold value to convert said input picture to a binary picture. 

28 A system according to claim 27, which further comprises means for determining said 

40 threshold value on the basis of an average value of the luminance from a plurality of comparison 40 
pictures between said input and reference pictures. f 

29 A system according to claim 1 , wherein said picture processing means comprises a 
diaphragm correcting means which provides an output thereof to said picture input means for 
correcting its diaphragm. . AK 

45 30 A system according to claim 29, wherein said diaphragm correcting means includes 

means for detecting a signal necessary for said diaphragm correction from signals of said input 
picture, means for setting a detection area for said signal detection, and diaphragm correction 
means for providing a diaphragm correction signal to said picture input means according to said 

signal obtained by said signal detecting means. ^ M en 

50 31. A system according to claim 30, wherein said signal detecting means comprises peak- s>u 

value detecting means for detecting the maximum luminance level. 

32. A system according to claim 30, wherein said signal detecting means comprises inte- 
qrated-value detecting means for detecting an average luminance level. 

33 A system according to claim 1 , wherein said picture processing means is provided for 
55 providing an output to a gate upon overflowing of an A/D converting means, said gate prov.des 

a clock signal to a counting means while the gate is made on, and said counting means prov.des 
an output to said abnormality discriminating means to stop said abnormality discrimination upon 
reaching a predetermined count. 

34 A system according to claim 1 , wherein said picture input means includes a p urality of 

60 picture pickup means, and means for switching outputs of said plurality of picture pickup means 60 
upon presence of luminance variation in said input picture. 

35 A system according to claim 34, wherein said switching means is a multiplexer. 

36' A system according to claim 1 , wherein said picture input means includes color-picture 
pickup means, and means for extracting only hue components from a picture signal output of 
65 said color-picture pickup means and providing said hue components to said picture processing bs 
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means 

37. An abnormality supervising system substantially as described herein with reference to the 
drawings. 
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